Portable device

ABSTRACT

A portable device, e.g. a laptop, includes a first part (110), e.g. a base element, and a second part (120), e.g. a lid element. The second part (120) contains an optical remote sensing system (300). The second part (120) is pivotably attached to the first part (110) via a hinge means (115), such that the portable device may be arranged in an open and a closed position respectively. The optical remote sensing system (300) is configured to track at least one distinctive feature of a user of the portable device when arranged in the open position. The first and second parts (110; 120) have a respective essentially flat inner surface (111; 121), which when the portable device is arranged in the closed position are parallel and face one another. The first part (110) further includes a recess (112a) which is arranged relative to a position of the optical remote sensing system (300) such that, in the closed position, the optical remote sensing system (300) is at least partly contained in the recess (112a).

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation application of and claims priority toand the benefit of U.S. application Ser. No. 15/259,538, filed Sep. 8,2016, which is a continuation application of and claims priority to andthe benefit of U.S. application Ser. No. 14/355,978, filed May 2, 2014and now U.S. Pat. No. 9,465,415, granted Oct. 11, 2016, whichapplication is further a national stage application, filed under 35U.S.C. § 371, of International Application No. PCT/EP2012/069445, filedOct. 2, 2012, which International Application further claims priority toand the benefit of U.S. Provisional Application Ser. No. 61/556,505,filed Nov. 7, 2011, and European Patent Office Application No.11187816.1, filed Nov. 4, 2011; the contents of all of which as arehereby incorporated by reference in their entirety.

BACKGROUND Related Field

The present invention relates generally to automatic tracking ofdistinctive features of users operating electronic equipment. Moreparticularly, the invention relates to a portable device according tothe claims provided herein.

Description of Related Art

Laptops with integrated eye-trackers are known. Unfortunately, the knownsolutions are comparatively bulky, and therefore the design becomesrelatively thick, i.e. in closed/inactive mode, the laptop has a ratherhigh profile. Naturally, this is undesired because portable devices,such as laptops, in general should be as compact and slim as possible.Nevertheless, bulkiness as such is associated with an advantage. Namely,the optical remote sensing systems of today's eye-trackers at timesconsume relatively large amounts of power. Therefore, substantialthermal power dissipation must be handled, and of course, in thiscontext, a larger volume is more useful than smaller ditto. The trendwhere each generation of laptop is thinner than the foregoing generationis problematic because the available space in the lid becomes verylimited. This places severe constraints on which components that can beused to implement eye-trackers and similar devices.

US 2005/0110887 shows an example of a mobile communication terminal witha main body and a foldable display body. A camera is here positioned ina hinge unit interconnecting the display body and the main body. Thecamera is rotatable around the hinge axis to register images of the useras well as subjects/objects in front of the user. Thus, a very versatilecamera function is attained. However, since the camera is freelyrotatable relative to both the main body and the display body it wouldbe complicated to use the camera for any purposes other than simpleimage registering, such as for eye-tracking.

In the light of the above, it is challenging to accomplish a compact andyet resourceful portable device that is equipped with an optical remotesensing system for eye-, gaze, gesture and/or facial feature trackingand/or user identification through face or iris recognition or handgesture detection.

BRIEF SUMMARY

The object of the invention is to mitigate the above problems andaccomplish a slim portable device with an integrated optical remotesensing system for eye- and/or gesture tracking.

According to the invention, the object is achieved by the initiallydescribed apparatus, wherein the first part includes a recess, which isarranged relative to a position of the optical remote sensing systemsuch that, in the closed position, the optical remote sensing system isat least partly contained in the recess.

This portable device is advantageous because it enables an overall slimdesign to be combined with an adequate cooling volume for the opticalremote sensing system. This is a desirable feature in any portabledevice, including: laptops, note books, ultrabooks, tablets withkeyboards, personal digital assistants and smartphones.

According to one preferred embodiment of the invention, the recess isco-located with a first piece of the hinge means. The optical remotesensing system is further disposed in a projection of the second partwhere a second piece of the hinge means is located. The first and secondpieces of the hinge means represent a pivot axis via which the first andsecond parts of the portable device are interconnected. Such anintegration of the optical remote sensing system in the hinge means isdesirable because it provides a volume sufficient to ensure adequatecooling of the optical remote sensing system while the trackingfunctionality can be integrated in an essentially indiscernible mannerinto the portable device.

According to another preferred embodiment of the invention, the firstand second parts of the portable device are interconnected via the hingemeans along a proximal side of the second part. The optical remotesensing system is here disposed in a projection extending along a distalside of the second part, where the distal side is opposite to theproximal side. In a laptop implementation, this means that the opticalremote sensing system is located above the screen when the device isarranged in the open/active mode. Such a positioning is especiallyadvantageous if a user's gestures are to be interpreted by the opticalremote sensing system.

According to yet another preferred embodiment of the invention, thesecond essentially flat surface of the second part includes a displayunit (e.g. an LCD screen) configured to present information to the user.It is further preferable if the first essentially flat surface of thefirst part includes a keyboard configured to receive input commands fromthe user. Hence, the optical remote sensing system is included in thesame part as the display unit.

According to still another preferred embodiment of the invention, theoptical remote sensing system is arranged such that a view angle thereofhas a fixed spatial relation to the display unit irrespective of anorientation of the second part relative to the first part. Of course,this is desirable because thereby it is fairly straightforward todetermine the user's point of regard on the display unit based on dataregistered by the optical remote sensing system.

According to a further preferred embodiment of the invention, the firstpart is a base element and the second part is a lid element. Duringoperation, the base element is configured to be placed on an essentiallyflat supporting surface (e.g. a desk) while the lid element ispositioned upright, so that its essentially flat inner surface(typically containing a display unit) is visible to the user.

According to other preferred embodiments of the invention, the opticalremote sensing system includes an image registering unit (e.g. a stillcamera or a video camera), and preferably, at least one illuminatorconfigured to illuminate the user. A combined camera-and-illuminator isgenerally advantageous for cost efficiency. In eye-trackerimplementations it is also desirable that one or more light sources bearranged close to the optical axis of the image registering unit. Theinvention, however, is likewise applicable to designs where the lightsource and the image registering unit are separated from one another. Inany case, it is generally preferable that the optical remote sensingsystem includes an optical filter, which is arranged in front of anilluminator and/or an image registering unit therein, and which opticalfilter is configured to block visible light however is transparent tonear-infrared (NIR) light. Namely, as will be discussed below, NIR lightis desirable, whereas visible light may disturb the user.

According to still another preferred embodiment of the invention, atleast one of the at least one illuminator is configured to producestructured light, which when registered by the image registering unit,causes resulting data to be created, which resulting data are adaptedfor generating a depth map of the user. This is advantageous both wheninterpreting gestures and in eye-tracking, for instance when selecting arelevant image segment to process.

According to yet another preferred embodiment of the invention, at leastone of the at least one illuminator is configured to producenear-infrared light. Namely, this type of light is relativelyuncomplicated to detect by a camera, however invisible to the human eye.

It is further preferred that at least one of the at least oneilluminator is configured to produce a light beam whose direction iscontrollable, so that a varying position of the user can be tracked.Directional optical illuminators are advantageous relative to staticditto because, at each point in time, the directional illuminator onlyilluminates a fraction of a surface inside a volume within which thesubject moves. Thereby, power is conserved corresponding to the size ofthe non-illuminated surface that would otherwise have been illuminated.

According to another preferred embodiment of the invention, at least oneof the at least one illuminator is based on LED (Light Emitting Diode)technology. Namely, LEDs represent energy-efficient, compact andreliable light sources.

According to still another preferred embodiment of the invention, atleast one of the at least one illuminator is configured to producecoherent light. Coherent light sources (e.g. lasers) are desirable,since such a light source may be combined with diffractive opticalelements to transform a light beam into a desired spatial pattern. Thus,the illumination can be controlled very efficiently, for instance tofollow a position of the user.

According to a further preferred embodiment of the invention, theoptical remote sensing system includes an eye tracker configured torepeatedly determine a position of at least one eye of the user and/orrepeatedly determine a point of regard of the user relative to theportable device. Thereby, it is possible to generate input commands to alaptop based on the user's ocular activity.

Further advantages, beneficial features and applications of the presentinvention will be apparent from the following description and thedependent claims.

BRIEF DESCRIPTION OF THE FIGURES

The invention is now to be explained more closely by means of preferredembodiments, which are disclosed as examples, and with reference to theattached drawings.

FIGS. 1a-b show side views of a portable device according to a firstembodiment of the invention;

FIG. 2 shows a top view of a first part of the portable device in FIGS.1a and 1 b;

FIGS. 3a-b show side views of a portable device according to a secondembodiment of the invention;

FIG. 4 shows a top view of a first part of the portable device in FIGS.3a and 3 b;

FIG. 5 illustrates, in further detail, the first embodiment of theinvention depicted in FIGS. 1a, 1b and 2; and

FIG. 6 illustrates, in further detail, the second embodiment of theinvention depicted in FIGS. 3a, 3b and 4.

DETAILED DESCRIPTION OF VARIOUS EMBODIMENTS

We refer initially to FIGS. 1a and 1b , which show side views of aportable device including a remote optical sensing system 300 accordingto a first embodiment of the invention. For convenience, the portabledevice is here embodied as a laptop. However, according to theinvention, the portable device may equally well be represented by analternative electronic device, such as a note book, an ultrabook, atablet with a keyboard, a personal digital assistant (PDA) or asmartphone.

The proposed portable device includes a first part 110 (here representedby a laptop base element) and a second part 120 (here represented by alaptop lid element). The second part 120, in turn, includes the opticalremote sensing system 300. As is common in laptops, the second part 120is pivotably attached to the first part 110 via a hinge means 115, suchthat the portable device may be arranged in an open and a closedposition respectively. FIG. 1a illustrates the closed position(predominantly used when the device is inactive) and FIG. 1b illustratesthe open position (the typical position when the device is activated).The optical remote sensing system 300 is configured to track at leastone distinctive feature of a user of the portable device when the deviceis active and arranged in the open position. The at least onedistinctive feature may be an eye, a facial feature and/or a limb of theuser, such as his/her hand. Thereby, the optical remote sensing system300 may be used for eye-, gaze, gesture and/or facial feature trackingand/or user identification through face or iris recognition or handgesture detection.

The first and second parts 110 and 120 have a respective essentiallyflat inner surface 111 and 121. When the portable device is arranged inthe closed position, the essentially flat inner surfaces 111 and 121 areparallel and face one another, as can be seen in FIG. 1a . The innersurfaces 111 and 121 are not entirely flat. Namely, for example, thefirst part 110 has a recess 112 a in the essentially flat inner surface111, which recess 112 a is arranged relative to a position of theoptical remote sensing system 300 in the second part 120, such that, inthe closed position, the first part 110 receives a projection 125 a ofthe second part 120, which projection 125 a includes the optical remotesensing system 300. As a result, the optical remote sensing system 300is at least partly contained in the recess 112 a when the portabledevice is arranged in the closed position (i.e. normally the inactivemode).

The first essentially flat surface 111 of the first part 110 preferablyalso includes a keyboard configured to receive input commands from theuser. Moreover, the second essentially flat surface 121 of the secondpart 120 preferably includes a display unit 122 (see FIG. 5) configuredto present information to the user, such as text, graphics, imagesand/or video.

FIG. 2 shows a top view of the first part 110 of the portable device inFIGS. 1a and 1b . The recess 112 a preferably extends between a pair ofhinge means 115 (symbolically illustrated with dashed lines), which inturn are arranged in proximity to the portable device's sides. Above thehinge means 115, the second part 120 preferably extends to a full widthof the first part 110 (not shown).

As is apparent from FIGS. 1a, 1b and 2, the recess 112 a is co-locatedwith a first piece of the hinge means 115, say a female piece. A secondpiece of the hinge means 115, say a male piece, is located on theprojection 125 a of the second part 120. The first and second pieces ofthe hinge means 115 represent a pivot axis via which the first andsecond parts 110 and 120 are interconnected. Hence, as is shown in FIG.1a (and in further detail in FIG. 5), when the portable device isclosed, the projection 125 a is configured to swing down into the recess112 a, so that the recess 112 a houses the optical remote sensing system300, at least partly. It is technically possible that the recess 112 ais represented by a complete, or partial cut-out in the first part 110.The dashed line 113 in FIG. 2 shows an example of where an edge of sucha partial cut-out may be located. Nevertheless, it is desirable that abottom part of the recess 112 a covers at least any optical filter infront of an illuminator and/or an image registering unit in the opticalremote sensing system 300 when the portable device is closed. Foroptimal compactness of the portable device, the recess 112 a should beas deep as possible; i.e. in closed mode, as much as possible of theoptical remote sensing system 300 should be contained in the recess 112a (or 112 b, see below). This means that the bottom of the recess 112 amay be a 2 mm thick tongue covering a front-most side of the opticalremote sensing system 300 when the portable device is closed.

FIGS. 3a and 3b show side views of a portable device including anoptical remote sensing system 300 according to a second embodiment ofthe invention. FIG. 4 shows a top view of the first part 110 of theportable device in FIGS. 3a and 3b . In FIGS. 3a, 3b and 4 all elementshaving labels which also occur in FIGS. 1a, 1b and/or 2 represent thesame entities as those described above with reference to FIGS. 1a, 1band/or 2.

Again, the portable device has first and second parts 110 and 120 thatare pivotably attached to one another, such that the portable device maybe arranged in an open and a closed position respectively. In this case,however, the optical remote sensing system 300 is not co-located withthe hinge means 115. Instead, the optical remote sensing system 300 isdisposed in a projection 125 b extending along a distal side of thesecond part 120, whereas the hinge means 115 are arranged along aproximal side of the second part 120, which proximal and distal sidesare opposite to one another.

FIG. 5 illustrates in further detail the first embodiment of theinvention depicted in FIGS. 1a, 1b and 2. Here, the optical remotesensing system 300 is arranged in the projection 125 a of the secondpart 120, which is co-located with the hinge means 115. Preferably, theoptical remote sensing system 300 is further arranged such that a viewangle a thereof has a fixed spatial relation to the display unit 122irrespective of an orientation of the second part 120 relative to thefirst part 110. The ideal view angle depends on the specificcharacteristics of an image registering unit included in the opticalremote sensing system 300. However, preferably a has a value in therange 50° to 65°, more preferably 56° to 60°, and most preferably a<<58°. Namely, thereby the risk that the user's hands block the opticalremote sensing system 300 is relatively low, and at the same time, anacceptable angle towards the user's face is attained for the mostcommonly used inclinations of the second part 120 relative to the firstpart 110 in the active mode. Hence, for example the user's point ofregard on the display unit 122 can be determined repeatedly based ondata from an eye-tracker of the optical remote sensing system 300.

FIG. 6 illustrates in further detail the second embodiment of theinvention depicted in FIGS. 3a, 3b and 4. Here, the optical remotesensing system 300 is arranged in the projection 125 b of the secondpart 120, which extends along a side of the second part that is oppositeto the side where the hinge means 115 are located. Also in thisembodiment, the optical remote sensing system 300 is preferably arrangedsuch that a view angle a thereof has a fixed spatial relation to thedisplay unit 122 irrespective of an orientation of the second part 120relative to the first part 110. Nevertheless, in this case, the viewangle a preferably has a value in the range 80° to 100°, and mostpreferably a <<90°. Thus, the optical remote sensing system 300 may beefficiently used for eye- and/or gaze tracking as well as forinterpretation of facial expressions and/or gestures.

Irrespective of whether the optical remote sensing system 300 isco-located with the hinge means 115 (as in FIG. 5), or arranged inproximity to a distal side thereof (as in FIG. 6), it is generallydesirable that an illuminator and/or an image registering unit of theoptical remote sensing system 300 is covered by an optical filter,which, preferably, is configured to block visible light while permittingNIR light to pass through.

Additionally, regardless of the location of the optical remote sensingsystem 300, according to preferred embodiments of the invention, theoptical remote sensing system 300 includes an image registering unit andat least one illuminator configured to illuminate the user. The imageregistering unit, in turn, may contain a still and/or a video cameraconfigured to capture image data representing the user of the portabledevice, such as images of his/her eyes.

It is further advantageous if at least one of the at least oneilluminator is configured to produce structured light, which whenreflected against the user and registered by the image registering unitcauses resulting data to be created, which resulting data are adaptedfor generating a depth map of the user. Depth maps are advantageous bothwhen interpreting gestures and during eye-tracking, for instance whenselecting a relevant image segment to process.

Moreover, one or more of the at least one illuminator may be configuredto produce (NIR) light. NIR light is advantageous because it isrelatively uncomplicated to detect by a camera and because it isinvisible to the human eye. Thus, NIR light does not disturb the user.

It is further advantageous if one or more of the at least oneilluminator is configured to produce a light beam whose direction iscontrollable to track a varying position of the user. If at least one ofthe at least one illuminator is configured to produce coherent light,diffractive optical elements (DOE) may be used to transform the lightbeam into a desired spatial pattern. Thus, the illumination can becontrolled very efficiently, for instance to follow a position of theuser.

Alternatively, or as a complement, at least one of the at least oneilluminator may be based on LED technology. LEDs are desirable lightsources since they are energy-efficient, compact and reliable.

The term “comprises/comprising” when used in this specification is takento specify the presence of stated features, integers, steps orcomponents. However, the term does not preclude the presence or additionof one or more additional features, integers, steps or components orgroups thereof.

The invention is not restricted to the described embodiments in thefigures, but may be varied freely within the scope of the claims.

The invention claimed is:
 1. A portable device comprising: a first part(110); and a second part (120) comprising an optical remote sensingsystem (300), wherein: the second part (120) is pivotably attached tothe first part (110) via a hinge means (115) such that the portabledevice may be arranged in an open and a closed position respectively;the optical remote sensing system (300) is configured to determine anidentity of a user of the portable device when the device is arranged inthe open position; the first and second parts (110; 120) have respectiveessentially flat inner surfaces (111; 121), which when the portabledevice is arranged in the closed position are parallel and face oneanother; and the first part (110) comprises a recess (112 a; 112 b),which is arranged relative to a position of the optical remote sensingsystem (300) such that, in the closed position, the optical remotesensing system (300) is at least partly contained in the recess (112 a;112 b).
 2. The portable device according to claim 1, wherein the opticalremote sensing system (300) is configured to determine the identity ofthe user of the portable device at least based on iris recognition. 3.The portable device according to claim 1, wherein the optical remotesensing system (300) is configured to determine the identity of the userof the portable device at least based on remote gesture recognition. 4.The portable device according to claim 1, wherein the recess (112 a) isco-located with a first piece of the hinge means (115) and the opticalremote sensing system (300) is disposed in a projection (125 a) of thesecond part (120) where a second piece of the hinge means (115) islocated, the first and second pieces defining a pivot axis of the hingemeans (115) via which the first and second parts (110; 120) of theportable device are interconnected.
 5. The portable device according toclaim 1, wherein: the first and second parts (110; 120) of the portabledevice are interconnected via the hinge means (115) along a proximalside of the second part (120); and the optical remote sensing system(300) is disposed in a projection (125 b) extending along a distal sideof the second part (120), the distal side being opposite to the proximalside.
 6. The portable device according to claim 1, wherein the secondessentially flat surface (121) of the second part (120) includes adisplay unit (122) configured to present information to the user.
 7. Theportable device according to claim 6, wherein the optical remote sensingsystem (300) is arranged such that a view angle thereof has a fixedspatial relation to the display unit (122) irrespective of anorientation of the second part (120) relative to the first part (110).8. The portable device according to claim 1, wherein the firstessentially flat surface (111) of the first part (110) includes akeyboard configured to receive input commands from the user.
 9. Theportable device according to claim 1, wherein the first part (110) is abase element and the second part (120) is a lid element, and duringoperation, the base element is configured to be placed on an essentiallyflat supporting surface while the lid element is positioned so that itsessentially flat inner surface (121) is visible to the user.
 10. Theportable device according to claim 1, wherein the optical remote sensingsystem (300) comprises an image registering unit configured to registerimage data representing the user.
 11. The portable device according toclaim 10, wherein the optical remote sensing system (300) comprises atleast one illuminator configured to illuminate the user with structuredlight, which when registered by the image registering unit, createsresulting data adapted for generating a depth map of the user.
 12. Theportable device according to claim 10, wherein the optical remotesensing system (300) comprises at least one illuminator configured toproduce near-infrared light.
 13. The portable device according to claim10, wherein the optical remote sensing system (300) comprises at leastone illuminator configured to produce a light beam whose direction iscontrollable to track a varying position of the user.
 14. The portabledevice according to claim 10, wherein the optical remote sensing system(300) comprises at least one illuminator which is based on LEDtechnology.
 15. The portable device according to claim 10, wherein theoptical remote sensing system (300) comprises at least one illuminatorwhich is configured to produce coherent light.
 16. The portable deviceaccording to claim 1, wherein the portable device is at least oneelectronic device selected from a group consisting of: a laptop, a notebook, an ultrabook, a tablet with a keyboard, a personal digitalassistant, and a smartphone.
 17. The portable device according to claim1, wherein the optical remote sensing system (300) comprises an eyetracker configured to at least one of repeatedly determine a position ofat least one eye of the user or repeatedly determine a point of regardof the user relative to the portable device.
 18. A method comprising thesteps of: providing a portable device comprising first and second parts(110; 120), the second part (120) comprising an optical remote sensingsystem (300), and the second part (120) being pivotably attached to thefirst part (110) via a hinge means (115) such that the portable devicemay be arranged in an open and a closed position respectively, the firstand second parts (110; 120) having respective essentially flat innersurfaces (111; 121), which when the portable device is arranged in theclosed position are parallel and face one another; and the first part(110) comprising a recess (112 a; 112 b), which is arranged relative toa position of the optical remote sensing system (300) such that, in theclosed position, the optical remote sensing system (300) is at leastpartly contained in the recess (112 a; 112 b); and determining anidentity of a user of the portable device, via the optical remotesensing system (300), when the portable device is arranged in the openposition.
 19. The method according to claim 18, wherein the identity ofthe user of the portable device is determined via iris recognition. 20.The method according to claim 18, wherein the identity of the user ofthe portable device is determined via remote gesture recognition. 21.The method according to claim 18, wherein: the second essentially flatsurface (121) of the second part (120) includes a display unit (122);and the method further comprises the step of presenting information tothe user via display unit (122) when the portable device is arranged inthe open position.
 22. The method according to claim 21, wherein: theoptical remote sensing system (300) comprises at least one illuminator;and when the portable device is arranged in the open position, themethod further comprises the steps of: illuminating the user withstructured light from the at least one illuminator; registering imagedata via the image registering unit; and based upon the registered imagedata; and creating resulting data configured for generating a depth mapof the user.
 23. The method according to claim 21, wherein: the opticalremote sensing system (300) comprises at least one illuminator; and whenthe portable device is arranged in the open position, the method furthercomprises the step of illuminating the user with near-infrared lightfrom the at least one illuminator.
 24. The method according to claim 21,wherein: the optical remote sensing system (300) comprises at least oneilluminator; and when the portable device is arranged in the openposition, the method further comprises the step of producing a lightbeam whose direction is controllable to track a varying position of theuser.
 25. The method according to claim 18, wherein: the firstessentially flat surface (111) of the first part (110) includes akeyboard; and the method further comprises the step of receiving inputcommands from the user via the keyboard when the portable device isarranged in the open position.
 26. The method according to claim 18,wherein: the optical remote sensing system (300) comprises an imageregistering unit; and the method further comprises the step ofregistering image data representing the user via the image registeringunit when the portable device is arranged in the open position.
 27. Themethod according to claim 18, wherein: the optical remote sensing system(300) comprises an eye tracker; and when the portable device is arrangedin the open position, the method further comprises the steps ofdetermining, repeatedly, a position of at least one eye of the user. 28.The method according to claim 18, wherein: the optical remote sensingsystem (300) comprises an eye tracker; and when the portable device isarranged in the open position, the method further comprises the steps ofdetermining, repeatedly, a point of regard of the user relative to theportable device.